The history of dynamic memory allocation can be traced back to around the 1950's [DSA-1995]. It gained significant popularity around the 1980's due to the UNIX operating system and the ‘C’ programming language. The early programming model in UNIX was based on a collection of documented calls which included functions such as ‘calloc’, ‘malloc’, ‘realloc’ and ‘free’. Later, a number of other operating systems and programming languages extended and modified these concepts, such as the ‘new’ and ‘delete’ operators in ‘C++’.
The history of storage management can be traced back even further than the origins of dynamic memory allocation. Again, many of the concepts and models at the time of writing became popular around the 1980's due to the UNIX operating system. The early programming model in UNIX was based on a collection of documented calls which included functions such as ‘open’, ‘read’, ‘write’ and ‘close’. Again, a number of later operating systems extended and modified these concepts, such as Linux and Microsoft Windows.
The history of computer oriented databases can be traced back to around the 1960's and appears to have been related to the introduction and use of direct access storage [HOD-2013]. In particular, the concept of transactions was introduced into databases to ensure that data on backing storage devices could be reliably modified even in the presence of common hardware and software failures (i.e. these are often termed ‘ACID’ properties). An aspect of various embodiments of the present invention is to transactionally allocate space in main memory and files or storage devices at the same time. Such transactional allocation may help to improve the reliability of the operating system and/or programs, due to the reduction in complexity as a result of the transactional allocation.
A program that uses the classical UNIX file management calls such as ‘open’, ‘read’, ‘write’ and ‘close’ typically operates in the following way. First it will ‘open’ the required file. Next, it will allocate some main memory space using a call such as ‘malloc’. Next it will ‘read’ the required part of the file into this main memory area. If this information needs to be changed it will then update the main memory area as necessary and ‘write’ the updated contents of the main memory area back to the file. Next, it may optionally release the main memory area using a call such as ‘free’. Some of these steps may be repeated multiple times if necessary. When the file is no longer needed a call to ‘close’ is used to release the file.
A significant area of difficulty in a program of this type is that it needs to decide where to place any new data that is added into the file. This is not a problem with main memory allocation as calls such as ‘malloc’ and ‘realloc’ automatically do this placement and return a pointer to the allocated space in main memory. A caller does not need to concern themselves with questions about how this is achieved. A caller can simply use the allocated main memory in any way that is necessary for as long as is required. Later, if an allocation is no longer needed it can easily be returned to the heap by a call to ‘free’. Such commands and their usage are known in operating systems such as UNIX.
Various embodiments described herein extend the concepts of main memory allocation to files and storage devices so as to simplify file and storage management. In many cases this simplification not only reduces complexity but also improves performance and reliability.
The above has been described with reference to exemplary advantages that are apparent when various embodiments of the invention are performed at an operating system level. Nonetheless, various embodiments of the invention are equally applicable to a wider range of storage devices and related concepts. A number of operating systems (e.g. Microsoft Windows) allow direct access to storage devices from user mode programs. In such cases, various embodiments may be used directly with these devices without modification. In other cases, various embodiments can be integrated into a device driver or firmware to get the desired level of access to the necessary hardware. Consequently, we will use the terms of ‘storage area’ and ‘storage device’ from this point forward to highlight that various embodiments of the invention have applicability well beyond operating system files.
In order to reduce the complexities described above, according to an aspect of various embodiments of the invention there is provided a system and method as defined in the appended claims.